The Institutional Dimension of Document Quality Judgments

نویسندگان

  • Bing Bai
  • Kwong Bor
  • Ying Sun
  • Paul Kantor
  • Tomek Strzalkowski
چکیده

In addition to relevance, there are other factors that contribute to the utility of a document. For examples, content properties like depth of analysis and multiplicity of viewpoints, and presentational properties like readability and verbosity, all will affect the usefulness of a document. These kinds of relevance-independent properties are difficult to determine, as their estimations are more likely to be affected by personal aspects of one's knowledge structure. Reliability of judgments on those properties may decrease when one moves from the personal level to the global level. In this paper, we report several experiments on document qualities. In our experiments, we explore the correlation of judgments on nine different document qualities to see if we can generate fewer dimensions underlying the judgments. We also investigate the issue of reliability of the judgments and discuss its theoretical implications. We find that, between the global level of agreement of judgment and the inter-personal level of agreement of judgment, there is an intermediate level of agreement, which can be characterized as an inter-institutional level.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Institutional Requirements for Enhancing Financial Stability in Iran

 The conditions in which the financial system can continuously provide the necessary services for the proper functioning of the economy is called “financial stability”. In this study, using global experience and Iranian data from 1990 to 2016, an aggregate index of financial stability for Iran has been constructed under the four dimensions of "deposit taker institutions", "macroeconomics", "for...

متن کامل

Crowdsourcing Relevance Assessments: The Unexpected Benefits of Limiting the Time to Judge

Crowdsourcing has become an alternative approach to collect relevance judgments at scale thanks to the availability of crowdsourcing platforms and quality control techniques that allow to obtain reliable results. Previous work has used crowdsourcing to ask multiple crowd workers to judge the relevance of a document with respect to a query and studied how to best aggregate multiple judgments of ...

متن کامل

The Role of Institutional Quality in Forecasting the Total Stock Price Index: Case Study of Developing and Developed Countries

The purpose of this paper is the empirically examination of the effect of the Institutional Quality Components (Voice and Accountability, political stability and Absence of Violence and Terrorism, government effectiveness, regulatory quality, rule of law and control of corruption) on the prediction of total stock price index. The study was carried out using data extracted from 53 countries from...

متن کامل

Inflation Bias, Time Inconsistency of Monetary and Fiscal Policies and Institutional Quality

In developing countries, weak institutional quality can increase the probability of applying discretionary policies and can have a great impact on their double-digit inflation. Surico (2008) calculated inflation bias, but he considered just monetary policy and he did not pay attention to the institutions. Therefore, we design a model which considers the discretion in monetary and fiscal policie...

متن کامل

Managing the Quality of Large-Scale Crowdsourcing

Crowdsourcing can be used to obtain relevance judgments needed for the evaluation of information retrieval systems. However, the quality of crowdsourced relevance judgments may be questionable; a substantial amount of workers appear to spam HITs in order to maximize their hourly wages, and workers may know less than expert annotators about the topic being queried. The task for the TREC 2011 Cro...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2004